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(57) Abstract: A method for acquisition of motion capture data which is capable of reducing the amount of data indicative of mo- 
tions, readily expressing natural motions in image production fields such as game character production, digital animation production, 
etc. and enabling a data provider to act and move with no limitation in sphere because of the absence of cable connections to his body. 
A plurality of color pointers with different color values are attached respectively to main joints and terminal portions of the body 
of the data provider. Three digital cameras (100, 200 and 300) or more are used to shoot actions and motions of the data provider. 
Acomputer is used to convert images picked-up by the digital cameras into X, Y and Z coordinate values. The motions of the body 
are extracted as three-dimensional position coordinate values based on motions of the color pointers. 
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METHOD FOR ACQUISITION OF MOTION CAPTURE DATA 
Technical Field 

The present invention relates in general to the acquisition of motion 
capture data, and more particularly to a method for acquisition of motion capture 
5 data, wherein motions of a person or animal are captured using color markers and 
three-dimensional motion data is produced on the basis of the captured motions. 

Background Art 

With the development of motion picture and game industries, there have 
been introduced three-dimensional modeling, rendering and animation 
10 technologies taking a concentrated interest in a technique capable of producing 
natural motions. 

In order to meet such an interest, there has been proposed a motion 
capture technique which is capable of directly capturing motions of a person, 
numerically expressing the captured motions, entering the resulting motion data in 
15 a computer and moving a character, etc. on the basis of the entered motion data. 

In this motion capture technique, it is most important to rapidly acquire 
data regarding natural body motions. 

Most conventional motion capture systems are adapted to attach markers 
or sensors to the human body, analyze output data from the sensors or marker 
20 images picked-up by cameras and measure the positions and orientations of 
respective joints of the human body in accordance with the analyzed results. 

Such conventional motion capture systems may be classified into four 
types, an acoustic type, mechanical type, magnetic type and optical type, according 
to the operational types of markers or sensors used therein. Among them, the 
2 5 systems of the magnetic type and optical type are most widely used now. 

The above conventional motion capture systems of the various types have 
the following characteristics and problems. 
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The motion capture system of the acoustic type comprises a plurality of 
acoustic sensors and three acoustic receivers. The acoustic sensors are attached 
to respective joints of an actor to sequentially generate acoustic waves, and the 
acoustic receivers are disposed apart from the acoustic sensors to receive the 
5 acoustic waves generated by the sensors. This motion capture system is adapted 
to calculate distances from the acoustic sensors to the acoustic receivers using time 
periods required for the receivers to receive the acoustic waves from the sensors. 
The position of each of the sensors in a three-dimensional space is obtained on the 
basis of a triangular surveying principle using values calculated respectively by the 

1 0 three receivers. This system of the acoustic type is disadvantageous in that the 
sampling frequency is low and the number of acoustic generators available at the 
same time is small. Also, motions of the actor may become unnatural because the 
sensors are large in size. Moreover, the acoustic-type motion capture system is 
greatly influenced by acoustic reflex because of the properties of its acoustic 

15 arrangement. 

The motion capture system of the mechanical type comprises a 
combination of a potentiometer and a slider for measuring motions of joints of an 
actor. Because this motion capture system has no receiver, it provides an 
absolute measurement arrangement which does not suffer any environmental 

20 interference or effects. Accordingly, as compared with the system of the acoustic 
type, the system of the mechanical type scarcely requires an initial calibration and 
need not have high-cost studio equipment. Further, as compared with other 
motion capture systems, the mechanical-type system is low in cost and very high 
in sampling frequency. However, the motion capture system of the mechanical 

25 type has a disadvantage in that the actor may unnaturally move because of 
mechanical units attached thereto. Moreover, the measurement accuracy of the 
mechanical-type system depends on how accurately the mechanical units are 
positioned on the respective joints of the actor. 

The motion capture system of the magnetic type comprises a plurality of 

30 sensors attached to respective joints of an actor for sensing a magnetic field. 
When the actor moves around a magnetic field generator, the sensors sense 
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magnetic field variations, which are then transformed into spatial values used for 
motion measurement. The sensors, magnetic field generator and system body are 
interconnected via cables or, recently, wirelessly. The motion capture system of 
the magnetic type is advantageous in that it is relatively low in cost and excellent 
5 in cost-to-performance ratio and can perform real-time processing. However, in 
the case where the sensors are of a wired type, the actor must move within a 
limited sphere due to cable connections from the sensors to his or her body, 
thereby making it difficult for the actor to naturally express his or her complex and 
rapid motions. For a wireless system employing a wireless transmitter instead of 
10 the cables, the actor cannot naturally move, either, because of the large size of the 
transmitter attached to his or her body. Further, the actor must move within a 
limited sphere because he or she must remain within the range of the magnetic 
field. 

The motion capture system of the optical type comprises a plurality of 
15 reflex markers attached respectively to main joints of an actor for reflecting 
infrared rays, and a plurality of cameras, each having three to sixteen infrared 
filters mounted thereto. The cameras are adapted to create two-dimensional 
coordinates of reflected marker images. A dedicated program is used to analyze 
the two-dimensional coordinates captured by the cameras and transform them into 
20 three-dimensional spatial coordinates as a result of the analysis. This 
conventional optical-type motion capture system is disadvantageous in that it lays 
emphasis on the realistic expression of motions of a character, resulting in the need 
for an extraordinary large amount of data and a high-performance computer to 
process it. 

2 5 Disclosure of the Invention 

Therefore, the present invention has been made in view of the above 
problems, and it is an object of the present invention to provide a method for 
acquisition of motion capture data which is capable of processing a reduced 
amount of data by extracting and processing only coordinate values varying with 
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motions. 

It is another object of the present invention to provide a method for 
acquisition of motion capture data which is capable of very simply and 
conveniently performing an initial calibration for motion capture. 
5 It is a further object of the present invention to provide a method for 

acquisition of motion capture data wherein the amount of data to be processed is 
small and a low-cost personal computer, camcorder and USB camera are used with 
no need for complex or high-cost peripheral equipment, thereby implementing a 
more cost-effective system. 

10 It is another object of the present invention to provide a method for 

acquisition of motion capture data wherein no cable is connected between an actor 
and a motion data processing system, thereby making it possible to conveniently 
extract motions of the actor, including a violent action. 

It is yet another object of the present invention to provide a method for 

15 acquisition of motion capture data which is capable of accurately measuring 
motions of an actor irrespective of ambient environments such as an ambient 
illumination, radio interference, etc. 

In accordance with the present invention, the above and other objects can 
be accomplished by a provision of a method for acquisition of motion capture data 

20 in a motion capture system which is capable of recording motions of a person or 
object in a computer-processable form. This method is based on a variety of 
technical requirements, or a color extraction algorithm, a color position tracking 
algorithm, a technique for tracking motions of an actor using a plurality of color 
markers with different color values, an image processing technique for perceiving 

25 images of 24 to 30 frames per second by a USB camera, and a technique for 
transforming extracted two-dimensional data into three-dimensional data. 

This motion capture technique has a basic interface function of acquiring 
data regarding motions of a character prior to realistically expressing the motions 
of the character. 

30 In view of the fact that the motion capture technique is a kind of interface 

method, the present invention proposes a method capable of readily acquiring data 
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about motions of an actor. 

Brief Description of the Drawings 

The above and other objects, features and other advantages of the present 
invention will be more clearly understood from the following detailed description 
5 taken in conjunction with the accompanying drawings, in which: 

Fig. 1 is a view showing the construction of a system for execution of a 
method for acquisition of motion capture data in accordance with the present 
invention; 

Fig. 2 is a view showing an arrangement of cameras for acquisition of 
1 0 three-dimensional data in accordance with the present invention; 

Fig. 3 is a schematic view illustrating the detection of X and Z coordinate 
values by a front camera in Fig. 2; 

Fig. 4 is a schematic view illustrating the detection of X, Y and Z 
coordinate values by a left camera in Fig. 2; 
15 Fig. 5 is a schematic view illustrating the detection of X, Y and Z 

coordinate values by a right camera in Fig. 2; 

Fig. 6 is a view showing a virtual motion area of an object whose motions 
are to be captured; and 

Figs. 7 to 9 are views showing respective dead zones of the front, left and 
2 0 right cameras in Fig. 2 and illustrating the detection of data in the dead zones. 

Best Mode for Carrying Out the Invention 

As seen from Figs. 1 and 2, a plurality of color pointers with different 
color values are attached respectively to main joint portions and terminal portions 
of the body of a data provider. Three or more cameras are equiangularly arranged 
25 while being spaced apart from one another at regular intervals, to shoot actions and 
motions of the data provider. A personal computer is used to convert image data 
picked-up by the cameras into three-dimensional coordinate values (X, Y and Z), 
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detect position variations of the color pointers on the respective body portions of 
the data provider from the converted three-dimensional coordinate values and store 
the detected position variations as motion capture data. 

The computer is adapted to receive image signals from the cameras 
5 through its buffer and extract only position coordinate values about motions of the 
color pointers on the respective body portions of the data provider from the 
received image signals. Because the computer extracts and stores not all the 
image signals, but only position coordinate values varying with motions of the 
color pointers on the respective body portions, the amount of data to be processed 
10 is reduced, resulting in an increase in processing speed. 

Now, a description will be given of a procedure of capturing motions and 
extracting three-dimensional motion data from captured images in accordance with 
the present invention. 

The following Table 1 shows an example of the assignment of 13 primary 
1 5 colors among 256 colors to main body portions of an actor. 



[TABLE 1] 





BODY PORTION 


COLOR 






BODY PORTION 


COLOR 


1 


Head 


Color 1 


8 


Left hand 


Color 140 


2 


Left shoulder 


Color 20 


9 


Right hand 


Color 160 


3 


Right shoulder 


Color 40 


10 


Left knee 


Color 180 


4 


Waist 


Color 60 


11 


Right knee 


Color 200 


5 


Hip 


Color 80 


12 


Left foot 


Color 220 


6 


Left elbow 


Color 100 


13 


Right foot 


Color 240 


7 


Right elbow 


Color 120 









Herein, the resolution is set to 160 pixels x 120 pixels, the color depth is 
set to 16 bits, the frame rate is set to 30 frames/sec, and N represents the number of 
color pointers. 

2 0 Under the above conditions, a camera extracts three primary color 

elements, or red, green and blue elements, from input images and transfers them to 
a personal computer (PC). At this time, the amount of data being transferred 
from the camera to the PC is 160 pixels x 120 pixels x 16 bits x 30 frames/sec = 
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9216000 bits/sec because it can be expressed by a multiplication of the resolution, 
color depth and frame rate. Calculating this value in terms of bytes, the result is 
1 152000 bytes/sec. Furthermore, calculating it in terms of Kbytes, the result is 
1152 Kbytes/sec. In conclusion, data of 1152 Kbytes per second is transferred 
5 from the camera to the computer. 

Because the amount of data calculated in the above manner is concerned 
with each of the red, green and blue elements, the amount of data captured for all 
the elements is three times the calculated data amount, or 1152000 x3 = 3456000 
bytes/sec. Calculating it in terms of Mbytes, the result is 3.456 Mbytes/sec. 

10 Finally, the total amount of data to be processed in the computer is obtained by 
multiplying the amount of data captured for all the red, green and blue elements by 
the number of color pointers. 

Namely, the total amount of data to be processed in the computer is 3.456 
Mbytes/sec if N = 1 and 34.56 Mbytes/sec if N = 10. 

1 5 Coordinate systems shown in Figs. 1 to 9 are absolute coordinate systems 

available for computer three-dimensional (3D) graphic and animation 
technologies. Taking a front view of the coordinate systems shown in Figs. 1 to 
9, the Z axis is placed in the upward and downward directions, the X axis is placed 
in the left and right directions and the Y axis is placed in the forward and backward 

20 directions. 

Hereinafter, a detailed description will be given of a method for 
acquisition of motion capture data in accordance with the present invention in 
conjunction with the accompanying drawings. 

First, three or more cameras shoot actions and motions of a data provider 

25 and transfer the resulting images respectively to associated computers. The 
computers extract position variations of a plurality of color pointers, attached 
respectively to main joint portions and terminal portions of the body of the data 
provider, from the images transferred from the associated cameras and obtain two- 
dimensional motion data coordinate values from the extracted position variations. 

30 Subsequently, the computers transfer the obtained two-dimensional 

motion data coordinate values over an internal network to a server computer, 
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which then transforms the transferred coordinate values into three-dimensional 
motion capture data according to a synthesis program based on a three-dimensional 
synthesis function engine. 

Fig. 1 is a view showing the construction of a system for execution of a 
5 method for acquisition of motion capture data in accordance with the present 
invention. As shown in this drawing, three or more cameras are provided to 
shoot motions of a data provider and transfer the resulting images respectively to 
associated computers. The computers are adapted to process the images 
transferred from the associated cameras in the above calculation manner and 

10 transfer the processed results to a server computer over an internal network. The 
server computer is adapted to obtain three-dimensional motion capture data on the 
basis of data transferred from the above computers. 

Fig. 2 is a view showing an arrangement of cameras for acquisition of 
three-dimensional data in accordance with the present invention. As shown in 

15 this drawing, three cameras, or a front camera 100, left camera 200 and right 
camera 300 are disposed around an object, or a data provider. 

The three cameras are equiangularly arranged in such a manner that they 
are spaced apart from one another at regular angles of 120 degrees. A virtual area 
is set to a predetermined range around a cross point of views of the three cameras, 

2 0 and it is recognized as a coordinate system by computers. 

In the case where the three cameras are disposed as shown in Fig. 1, they 
acquire Z coordinate values on all sides of an object, X and Y coordinate values 
of the object are obtained by performing an appropriate calculation algorithm for X 
coordinate value variations acquired by the front camera 100 and +X, -Y, +Y and - 

25 Y data acquired by the left and right cameras 200 and 300. Motion capture data 
on all sides of the object can be acquired in this manner. 

Fig. 3 is a schematic view illustrating the detection of X and Z coordinate 
values by the front camera 100 in Fig. 2. All X and Z position coordinate values 
of pointers on the front side of an object are detected through the front camera 100 

30 and then provided as a reference for comparison with data detected through the left 
and right cameras 200 and 300. The X and Z coordinate values detected by the 
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front camera 100 are also provided as a reference for correction of data detected by 
the other cameras. 

Fig. 4 is a schematic view illustrating the detection of X (30°), Y (60°) 
and Z coordinate values by the left camera 200 in Fig. 2. Z coordinate values and 
5 X and Y coordinate values of pointers on the left and rear sides of an object are 
detected through the left camera 200. The detected X and Y coordinate values 
are rotated on the basis of the detected Z coordinate values and then provided as a 
reference for comparison with data detected by the front camera 100 and right 
camera 300. 

10 Fig. 5 is a schematic view illustrating the detection of Y (60°), X (30°) 

and Z coordinate values by the right camera 300 in Fig. 2. Z coordinate values 
and X and Y coordinate values of pointers on the right and rear sides of an object 
are detected through the right camera 300. The detected X and Y coordinate 
values are rotated on the basis of the detected Z coordinate values and then 

1 5 provided as a reference for comparison with data detected by the front camera 100 
and left camera 200. 

Fig. 6 is a view showing a virtual motion area (an X and Y coordinate 
system) of an object whose motions are to be captured. The virtual area is set to a 
predetermined range around a cross point of views of the three cameras 100, 200 

20 and 300, and all motions of the object are limited to within the virtual area. A 
dead zone of each of the cameras is detected through the crossing views of the 
other two cameras and accurate coordinate values thereof are obtained through 
verification and correction operations. 

The dead zones of the three cameras are shown respectively in Figs. 7 to 

25 9. 

Coordinate values of pointers positioned in the dead zone of the front 
camera 100 in Fig. 7 are determined by comparing data detected by the left and 
right cameras 200 and 300 with each other and verifying and correcting data 
detected by the front camera 100 on the basis of the compared result. 
30 Coordinate values of pointers positioned in the dead zone of the left 

camera 200 in Fig. 8 are determined by comparing data detected by the front and 
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right cameras 100 and 300 with each other and verifying and correcting data 
detecteid by the left camera 200 on the basis of the compared result. 

Coordinate values of pointers positioned in the dead zone of the right 
camera 300 in Fig. 9 are determined by comparing data detected by the front and 
5 left cameras 100 and 200 with each other and verifying and correcting data 
detected by the right camera 300 on the basis of the compared result. 

Industrial Applicability 

As apparent from the above description, the present invention provides a 
method for acquisition of motion capture data which is capable of extracting and 
1 0 processing only color pointer data representing motions of an actor. Therefore, 
the amount of data to be processed is significantly reduced, resulting in an increase 
in processing speed as compared with the processing of the entire image. Further, 
a low-cost personal computer, camcorder and USB camera are used with no need 
for complex or high-cost peripheral equipment, thereby implementing a more cost- 
15 effective system capable of very simply and conveniently performing an initial 
calibration for motion capture at any place. 

Further, there are no wired and wireless connections made between an 
actor and a motion data processing system, so that the actor can move with no 
limitation in sphere, thereby making it possible to conveniently extract motions of 
2 0 the actor, including a violent action. 

Moreover, the entire performance of the present system can be simpler 
improved by merely upgrading software, as compared with the conventional 
performance improvement based on hardware. 

Furthermore, owing to the above features, the present invention is widely 
25 applicable to an entertainment field, a virtual reality field and other image 
production fields. 

Although the preferred embodiments of the present invention have been 
disclosed for illustrative purposes, those skilled in the art will appreciate that 
various modifications, additions and substitutions are possible, without departing 
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from the scope and spirit of the invention as disclosed in the accompanying claims. 
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Claims: 

1. A method for acquisition of motion capture data, comprising the steps 

of: 

a) attaching a plurality of color pointers with different color values 
respectively to main body portions of an actor; 

b) shooting motions of said actor by camera means; and / 

c) converting image data picked-up by said camera means into three- 
dimensional coordinate values (X, Y and Z), detecting position variations of said 
color pointers on the respective body portions of said actor from the converted 
three-dimensional coordinate values and extracting three-dimensional position 
coordinate values based on motions of said color pointers from the detected 
position variations. 

2. A method for acquisition of motion capture data, as set forth in Claim 
1, wherein said camera means includes at least three cameras equiangularly 

15 arranged while being spaced apart from one another at regular intervals, said three 

cameras being a front camera, a left camera and a right camera disposed around 

said actor; and wherein said step c) includes the steps of: 

acquiring vertical motion (Z coordinate) data from all of said cameras, 

acquiring horizontal motion (X coordinate) data from said front camera and 
20 acquiring forward and backward motion data by performing a predetermined 

calculation algorithm for the acquired horizontal motion data and data acquired 

from said left and right cameras; and 

acquiring data regarding respective dead zones of said cameras by 

performing a parallel calculation algorithm for complementary data from said 
25 cameras, and generating three-dimensional data on the basis of the acquired dead 

zone data. 
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FIG. 8 
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